The need for producing good quality high resolution depth data is growing on variety of electronic devices including mobile, home entertainment, gaming, robots, drones, augmented or virtual reality (AR/VR) systems, cars, security, etc. The depth data is used in many imaging and detection applications in consumer and industrial markets.
Depth perceptive camera systems can be used to capture a scene and estimate the depth (or “z-distance”) of each pixel in a scene, thereby generating a “depth map.” Generally depth camera systems (or “depth cameras”) can be classified into passive depth cameras and active depth cameras.
Active depth cameras generally include an active illumination component or a projector which emits light onto a scene. Broadly, these include: “time-of-flight” active depth cameras, which emit diffuse modulated illumination onto the scene; and “structured light” active depth cameras, which emit an illumination pattern in order to project a pattern onto the scene. The pattern may be used in the determination of the disparities for general cases (e.g., by providing additional texture to low texture objects or by optimizing the determination of disparities), and also allows operation in conditions with insufficient ambient lighting (e.g., in dark environments). Passive depth camera systems lack such active illumination components.
Some depth camera systems utilize stereo vision techniques in which depth data is computed based on the disparity between matching features found in the images captured by multiple cameras. Using an image acquisition system that includes two or more cameras can improve robustness against deviations in the projection pattern and can also enable operation in situations with bright ambient lighting (e.g., in which a pattern projected by the active illumination component cannot be detected over the ambient light).
Some depth camera systems utilize stereo vision techniques in which depth data is computed based on the disparity between matching pattern features found in the images captured by one camera and a coded pattern projected in a so called structured light method. In the continuing text, the subject is described in the context of a more general multi-camera stereo system wherein the structured light method may be considered as a system sub-class.
When computing depth information, the depth camera system uses known calibration parameters, as discussed in more detail below. The calibration parameters illustrate that the depth calculations are sensitive to the distance between the cameras and their orientation with respect to one another (e.g., the angles between the optical axes or the optical planes of the cameras).